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Technological advances that involve human sensorimotor processes can have both 
intended and unintended effects on the central nervous system (CNS). This mini review 
focuses on the use of virtual environments (VE) to augment brain functions by enhancing 
perception, eliciting automatic motor behavior, and inducing sensorimotor adaptation. 
VE technology is becoming increasingly prevalent in medical rehabilitation, training 
simulators, gaming, and entertainment. Although these VE applications have often been 
shown to optimize outcomes, whether it be to speed recovery, reduce training time, 
or enhance immersion and enjoyment, there are inherent drawbacks to environments 
that can potentially change sensorimotor calibration. Across numerous VE studies over 
the years, we have investigated the effects of combining visual and physical motion on 
perception, motor control, and adaptation. Recent results from our research involving 
exposure to dynamic passive motion within a visually-depicted VE reveal that short-term 
exposure to augmented sensorimotor discordance can result in systematic aftereffects 
that last beyond the exposure period. Whether these adaptations are advantageous or not, 
remains to be seen. Benefits as well as risks of using VE-driven sensorimotor stimulation 
to enhance brain processes will be discussed. 
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INTRODUCTION 

The use of technology to augment brain function has a nebulous 
history, in part, because the term hrain augmentation can be 
defined in so many ways. It implies an enhancement of brain 
function, but how it is measured introduces some uncertainty. 
To clarify this further, it should be recognized that the function 
of the brain is nothing short of control of the entire body and 
all one's experiences, both internal processes and externally- 
directed actions. Therefore augmenting brain function could 
include a wide range of easy or difficult to identify enhancements, 
such as boosting immune system response, improving mood, 
memory, or perception, optimizing motor control, or increasing 
sub-optimal function after injury. To make brain augmentation 
even harder to define, sometimes the only measure we have of 
its effective implementation is through observation of external 
behavior. 

A simple definition of brain augmentation could be enhance- 
ment of sensation, such as the use of spectacles to improve 
myopia. Most glasses wearers will be familiar with the initial 
adaptation to a slight magnification and/or shift in close-up visual 
space that occurs with new glasses. This type of adaptation is 
even more evident when using light-refracting prism glasses to 
intentionally shift the visually perceived world. Facilitation of 
adaptive processes could be considered another form of brain 
augmentation. For example, prism adaptation has been used to 
correct strabismus by helping the eyes accomplish perceptual 



convergence and reducing the angle of squint (Pigassou, 1972). 
In hemispatial neglect wearing prism glasses can augment the 
brain's ability to adapt its sensorimotor representation of the 
external world and reduce the perceptual deficit (Rossetti et al., 
1998). 

Augmenting brain function by enhancing perception or com- 
pensating for a deficit is an area in which virtual reality (VR) 
technology is also being applied. Similar to prism adaptation, VR 
technology can be used to alter the egocentric and allocentric 
representations of the world (Castiello et al, 2004). In this paper, 
the issue of identifying enhancement in the realm of sensory 
perception, motor control, and sensorimotor adaptation by using 
virtual environments (VE) will be reviewed. The sensorimotor 
and perceptual processes that may cause these changes will be 
examined while also highlighting the difficulty that goes along 
with defining, measuring, and quantifying enhancement. Not all 
changes are enhancement, since some are not adaptive or are 
accompanied by unwanted degradation of other functions or 
cause unintended responses. 

AUGMENTING PERCEPTION 

A fundamental aspect of creating an immersive experience within 
a VE is in creating a sense of three-dimensionality so that one 
believes one can move about within the virtual world. A common 
barrier to this is that one may wish to travel further in the VE 
than is possible in a limited space (e.g., a flight simulation). 
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Although the perception of self-motion typically occurs as a 
result of combined visual and physical motion stimuli, its long 
been known that visually-induced self-motion perception (i.e., 
vection) can occur in the absence of actual motion (Mach, 1875; 
Young et al, 1973; Pavard and Berthoz, 1977; Howard, 1986; 
Frigon and Delorme, 1992; Harris et al., 2000; Palmisano, 2002; 
Wright, 2002). However, optic flow when walking about the real 
world is different than when rotating the eyes or head whUe the 
body remains stationary (Gibson, 1954). That our central nervous 
system (CNS) can accurately detect this difference likely evolved 
as an essential part of navigating through the world searching for 
food or mates and avoiding danger. The exact selective pressures 
that shaped these neural processes can only be inferred, how- 
ever, its very likely that in the pre-modern world earth-bound 
species (flightless and terrestrial) were less frequently exposed 
to the visual-vestibular discordance that occurs during passive 
motion to which we are often exposed in the modern world (e.g., 
during any vehicle transport). Although the multiple sensory 
inputs that occur during self-motion do not always have a pre- 
scribed relation, they co-occur with greater probability in certain 
relations to one another than others. For example, terrestrial 
animals experience sustained vection (very low frequency) in a 
direction orthogonal to vertical more frequently than parallel to 
it. Furthermore, the sensory organs are oriented upright relative 
to gravity more frequently than inverted. Despite this, it seems 
there was a strong selective advantage for omnidirectional sensory 
organs that could be mapped to each other rather flexibly. One 
need only don a pair of prism glasses to experience visuomotor 



remapping that makes even a completely inverted world even- 
tually feel "normal" (Stratton, 1896). This sensorimotor lability 
can be used to induce a compelling sense of immersion in a 
VE even when sensory inputs are incongruent, sub-threshold, or 
absent. 

The flexible, adaptable sensorimotor integration described 
above is how the brain's perceptual abilities can be augmented 
in a VE. For example, self-motion perception can be induced 
even when sensory input is below threshold. In a VE, the inertial 
stimulus is often sub-threshold or even absent, yet perception of 
self-motion is still experienced. In our VE studies, we have tested 
how this perception is affected by manipulating the concordance 
of visual and inertial sensory inputs (see Table 1; Wright, 2002, 
2009; Wright and Glasauer, 2003, 2006; Wright and Schneider, 
2009). To enhance the level of immersion, subjects viewed a high- 
fidelity, realistic visual scene from a first person perspective while 
wearing a head-mounted display (HMD; Wright et al, 2005, 
2006, 2009). To further enhance the perceptual experience, we 
drew on previously established knowledge about motion percep- 
tion. The visual system primarily transduces velocity information 
from the visual stimulus, and is less sensitive to acceleration 
(Berthoz et al, 1975; Telford et al, 1992; Warren and Kurtz, 
1992). Acceleration information is primarily derived from the 
inertial stimulus, which even in the absence of vision plays 
an important part of the experience of motion (Wright et al., 
2005). Gravitoinertial input resulting from physical motion and 
background gravity stimulate the vestibular and somatosensory 
systems (i.e., the graviceptors) which provide linear acceleration 



Table 1 | Perceptual and/or motor behavior In Immersive VE. 

Study Visual stimulus Physical stimulus 



Instructed response (DV) Unintended response 



Wright and Glasauer (2003) 
Wright etal. (2005) 

Wright etal. (2009) 

Wright and Schneider (2009) 

Wright (2009) 

Wright etal. (2013) 
Wright (2013) 



Static or Roll tilting VE 
scene 



Up-down translation of VE 
scene depicting various 
amplitudes of motion 

Left-right translation of VE 
scene 

Left-right or up-down 
translation of VE scene 

Left-right translation of VE 
scene 



Left-right translation of VE 
scene 

Fore-aft translation of VE 
scene in 3-walled, 
earth-fixed cave 



Roll tilt of whole body 
while sitting 



Up-down whole body 
translation either matching 
or mismatching visual 
amplitude 

Roll tilt of whole body 
while sitting 

None 



Up-down whole body 
translation of various 
amplitudes 

Fore-aft whole body 
translation while sitting 

Left-right whole-body 
translation while standing 



Subjective vertical 
(joystick/glass orientation) 



Subjective estimate of 
self-motion amplitude 



Subjective vertical (joystick 
orientation) 

Subjective vertical (joystick 
orientation) 

Subjective estimate of 
self-motion amplitude 



Stabilize head (yaw, pitch, 
roll angular velocity) 

Maintain balance (center 
of pressure) 



Motor response depends 
on type of object wielded 
even if similar size and 
weight 

Visual-vestibular weighting 
not always linear 
summation 



Joystick Translations 



Joystick Translations 



Left-right self-motion 
perception increased as 
physical up-down motion 
increased 

Yaw head movement 



Postural response to 
visual+physical stimulus. 
Postural aftereffects during 
post-adaptation visual 
stimulation 
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and rotation information (Guedry, 1974). But like the visual 
system, there is an overlap in what qualifies as an adequate 
stimulus, as suggested by the fact that graviceptors may also be 
sensitive to velocity cues (Jones and Young, 1978). Our investi- 
gations suggest that combinations of physical and visual stimu- 
lation can result in sensory summation. This can increase how 
compelling perceived self-motion is when stimuli are matched 
directionally and temporally (Wright et al., 2005). However, we 
have also found that perceived self-motion can be enhanced by 
adding together discordant stimuli, such as by combining physical 
tilt with visually-depicted translation (Wright et al., 2009). In 
fact, the "compellingness" of visually-induced self-motion can be 
enhanced in the direction of the visual stimulus by increasing 
the amplitude of a discordant inertial input. This is true even 
if the visual and inertial vectors are 90-180° out of alignment 
with each other (Wright et al, 2005; Wright, 2009; Ash et al, 
2011). 

Another contributing factor that augments the VE-user's 
immersion by enhancing the self-motion experience is predicated 
on the idea of tilt-translation ambiguity (Young et al, 1984; 
Parker et al, 1985; Merfeld et al, 2005; Holly et al, 2006). 
Accurate perception of tilt versus translation is obfuscated in 
part because gravitational and inertial accelerations are physically 
indistinguishable, and in part because the adequate stimulus 
for the visual system is velocity-dependent while the adequate 
stimulus for graviceptors is acceleration-dependent. Thus, tilt in 
a gravitational field could conceivably be perceived as translation. 
However, when multiple sensory inputs are combined, a unified 
sense of self-motion can be derived. Whether this perception is 
rotation or translation or some combination of both is, in part, 
determined by spatial, temporal, and cognitive factors (Wertheim 
et al, 2001; Wright et al., 2006; Holly and McCoUum, 2008; 
Riecke, 2009). The perception also depends on the independence 
of the velocity and acceleration vectors. In nature, a positive 
velocity could be experienced with an infinite number of posi- 
tive or negative acceleration vectors. This is important because 
the visual system's velocity-sensitive and the vestibular system's 
acceleration-sensitive. All these factors contribute to how VE can 
be used to induce unusual self-motion perceptions, despite the 
fact that the sensory pattern has never been experienced (Wright, 
2009). This fact has been capitalized on by IMAX®-based rides 
for decades, where an individual can experience compelling self- 
motion while completely stationary; moreover, the perception can 
be further enhanced when subtle movements of the theater seat 
are added. 

The phenomenological evidence above suggests that veridi- 
cal self-motion perception is dependent on the integration of 
multiple sensory inputs that do not have a prescribed relation 
to one another. Moreover, perception can be extended beyond 
the normal limits of the senses by combining sensory inputs in 
VE to induce non-veridical self-motion perception. A sense of 
immersion in a VE can be enhanced by the belief that movement 
within the VE is actually happening. But if the VE is augmenting 
perception of a "false" reality, does this also alter one's actions? 
And more importantly, is there a real-world benefit? The first 
question has been brought up before (Stanney et al., 1998; Cohn 
et al., 2000) but lately the focus of attention has shifted to the 



latter question. Many researchers and clinicians have embarked 
upon systematic investigation of how VR can augment the brain's 
ability to recover from disease and injury (e.g., Liebermann et al., 
2012). In the next sections we delve into these questions, first by 
determining whether motor behaviors are altered by VE immer- 
sion then by looking at sensorimotor adaptation within a VE. 

THE PERCEPTION-ACTION LINK IN VE 

Perception can be defined as the organization of sensory stimuli 
(both externally received or internally generated) resulting in a 
conscious awareness of a phenomenon. Perception and action are 
tightly linked processes, with the interaction of sensory input and 
motor output being fundamental to the calibration of fine and 
gross motor skills. Accurate perception of the body's movements 
is critical to fine-tuning motor skills. However, there are many 
examples of dissociations between perception and action (e.g., 
Goodale and Milner, 1992; Merfeld et al, 2005). Thus, even 
though perception of self-motion and orientation can be altered 
in a VE, how it affects motor output has required investiga- 
tion. In a series of studies, we looked at manual motor control, 
head stabilization, and postural control while immersed in a VE 
to evaluate whether the perception-action link reliably occurs 
in VE. 

Much like in the perceptual studies, we tested conditions that 
combined visually-depicted motion both with and without actual 
physical motion. Subjects performed a manual motor control task 
while sitting in a tiltable motion device and viewing a visual depic- 
tion of linear translation in an HMD (Wright and Glasauer, 2003, 
2006; Wright et al, 2009; Wright and Schneider, 2009). During 
this VE immersion, subjects were instructed to align a handheld 
object to the perceived vertical. Perceptual reports confirmed self- 
translation to be more compelling when physical tilt of the subject 
was added to the linear visual motion (Wright et al., 2009; Wright 
and Schneider, 2009). But interestingly, despite being asked only 
to align the handheld object to vertical, subjects also showed 
automatic, unconscious translation of the unconstrained arm. 
These unintended motor actions were entrained with the visually- 
depicted motion, as if experiencing accelerations due to actual 
translation (see Table 1). Although its been shown that constant 
velocity visual rotation in a VE can affect the direction of reach- 
ing (Cohn et al., 2000; Dvorkin et al, 2009), our findings also 
revealed that vision could induce vection with both velocity and 
acceleration components. Automatic motor responses occurred 
as if a compensatory force was required to counter the illusory 
acceleration. Moreover, the more compelling the perceived self- 
motion, the larger the motor response. 

We were also interested in determining how VE immersion 
might affect other motor control processes such as head stabi- 
lization (Wright et al., 2013) and whole-body postural control 
(Wright, 2013). In one study, subjects were exposed to physi- 
cal accelerations while viewing a directionally discordant visual 
scene. Significant head stabilization responses were dependent 
on the direction of visual flow even though visual input was 
discordant with the physical stimulus (Wright et al., 2013). In 
another study applying a similar experimental test protocol to 
investigate posture control, we paired visually-depicted transla- 
tion in one direction with an orthogonally directed translation 
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of the support surface. This cross-axis stimulation resulted in a 
postural response in standing humans that showed entrainment 
with both the visual and the support surface stimuli (Wright, 
2013). These findings may not be unexpected, since studies pre- 
dating VR technology have shown that postural control can 
be entrained to a visual stimulus by simulating movement of 
the entire room, when the floor is kept stationary (Lishman 
and Lee, 1973). Furthermore, the postural response can be 
influenced by the characteristics of the discordant visual input 
(Lestienne et al., 1977). If an immersive VE is used the postural 
response to a visual input can be potentiated even if the input 
is temporally discordant with surface movement (Keshner et al., 
2004). Together these results suggest that VE can be used to 
induce a motor response to a non-veridical virtual input, even 
in the presence of a strong destabilizing physical stimulus that 
requires an appropriate postural response to keep from falling. 
Thus, VE can cause an unintended postural response leading to 
instability. 

INDUCING SENSORIMOTOR ADAPTATION AND 
MALADAPTIVE AFTEREFFECTS 

To study postural adaptation in VE, we exposed subjects to cross- 
axis postural stimulation for an extended period of time. By 
looking for aftereffects, as is often done in prism-adaptation 
research, we wished to see if the postural response seen during 
discordant stimulation involved adaption or was simply a tran- 
sient response that would disappear as soon as one of the stimuli 
was removed (Wright, 2013). The plasticity of the CNS, which 
allows discordance to be reduced by sensorimotor re-mapping, 
is present at birth and plays an essential role in shaping motor 
development through interaction with the environment (field 
and Freedman, 1963). Seminal work on visuospatial adaptation 
using light-refracting prism glasses dating back a century and a 
half (von Helmholtz, 1867; Stratton, 1896) has shown us that 
although neural plasticity may diminish with age, this form of 
plasticity does not have a critical period; rather to some extent 
it lasts well into adulthood (Hardt et al., 1971). Therefore, deter- 
mining whether VE exposure will automatically cause sensori- 
motor adaptation is important. If one considers VE usage in 
everyday life, for training purposes, or for rehabilitation, one 
might expect very large exposure times, which could have long 
lasting aftereffects. 

We exposed standing subjects for an extended period of time 
(5 min) to the cross-axis stimulation. After the adaptation period, 
we stopped the dynamic support stimulation while subjects con- 
tinued to view the same visual translation stimulus as before. 
What was found in the post-adaptation phase is that subjects 
indeed showed an aftereffect in the COP, which was rotated 
as much as 45° from the direction of the visual stimulation. 
This finding is complementary to the finding which showed 
sensorimotor adaptation of the vestibulo-ocular reflex (VOR) in 
monkeys after prolonged pairing of physical rotation with an 
orthogonally directed optic flow (Wei and Angelaki, 2001). The 
evidence from our study on humans show that long lasting (>2 
min) whole-body sensorimotor adaption can also be induced 
after prolonged exposure (<5 min) to discordant combinations 



of inertial and visual stimulation in a VE (Wright, 2013). Prelim- 
inary evidence suggests that these adaptations may even last for a 
few days. Specifically, after a few exposures staggered over multi- 
ple days, aftereffects appear immediately upon re-immersion into 
the VE without requiring a fresh dose of cross-axis adaptation. 

This raises the question whether these residual aftereffects 
seen in healthy individuals who spend extended time in a VE 
are adaptive in the real world or unwanted aftereffects. Inaccu- 
rate automatic motor responses due to recalibration of visual, 
vestibular, and somatosensory coordination certainly involves a 
risk. However, for those with a perceptual or motor deficit, VE 
immersion may move them from a maladaptive state towards 
an adaptive one. Although this requires further investigation, 
what is apparent is that VE can be used to facilitate the brain's 
natural ability to adapt to changes in the sensory environment. 
In fact, VE-based brain augmentation applications have already 
begun to appear in clinical applications. They have been used to 
ameliorate hemi-neglect symptoms by remapping the neglected 
space (Castiello et al, 2004), thus reducing a perceptual deficit, 
and improving function. 

CONCLUSIONS 

The fidelity of immersive VE has increased with amazing speed 
over the last two decades. One need only play the latest release 
of any popular first person shooter or sports-themed video game 
to be astounded by its verisimilitude. Because of the level of 
immersion one can experience, the question arises as to how 
quickly our CNS adapts to such simulated reality. More impor- 
tantly, how robust are these adaptations, not only to the accurate 
attributes of the manufactured environments, but also to their 
inaccuracies. Even if visual fidelity such as spatial resolution, 
update rates, perspective geometry, monocular and binocular 
cues, et cetera are all perfectly programmed to match reality 
in a VE, the absence of inertial cues to accompany simulated 
translation through such environments will affect how the CNS 
calibrates the perceptuomotor system to these non-veridicalities. 
In other words, exposure to a VE will automatically cause sen- 
sorimotor adaptation, whether desired or not. But hopefully, 
VE's ability to augment brain functions by enhancing perception, 
eliciting automatic motor behavior, and inducing sensorimotor 
adaptation can be applied in useful ways. 

Trusting what we can measure as evidence of brain enhance- 
ment may not solve the problem of defining brain augmentation, 
when short-term benefits may accompany long-term deficits. 
Conversely, long-term enhancements may be overlooked because 
of a more immediate decrement in brain function that can happen 
as the brain adapts to or learns a new technology. What can 
be certain is that benefits from brain augmentation, if under- 
stood and applied, can be far-reaching. If used to recover loss 
of function following neurological damage, brain augmenta- 
tion can potentially bring function back to healthy levels. The 
ways in which scientist, technologist, and clinicians are apply- 
ing VR technology to rehabilitation is already happening and 
the hopes are that the individuals involved in this collective 
creative process will have the foresight to exploit the potential 
of neural plasticity, while recognizing that not all change is 
good. 
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